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Abstract —We consider a variable-length source coding prob¬ 
lem subject to local decodability constraints. In particular, 
we investigate the blocklength scaling behavior attainable by 
encodings of / -sparse binary sequences, under the constraint 
that any source bit can be correctly decoded upon probing at 
most d codeword bits. We consider both adaptive and non- 
adaptive access models, and derive upper and lower bounds 
that often coincide up to constant factors. Notably, such a 
characterization for the fixed-blocklength analog of our problem 
remains unknown, despite considerable research over the last 
three decades. Connections to communication complexity are also 
briefly discussed. 

Index Terms —Variable-Length Compression, Local Decoding, 
Bit-Probe, Static Membership 

I. Introduction 

Efficient representation of a sequence of source bits by a 
significantly shorter sequence of encoded bits (i.e., a code¬ 
word) is the classical problem of lossless source coding, 
proposed by Shannon in his seminal 1948 paper Q]. It is 
widely known that optimal compression performance can be 
achieved with schemes such as Huffman codes El or the 
Lempel-Ziv universal compression algorithm El- However, 
these compression schemes suffer from the drawback that they 
do not support local decodability. Specifically, retrieving a 
single bit of the source sequence generally requires a decoder 
to access all of the encoded bits. 

This is clearly undesirable in applications that favor retriev¬ 
ing selected pieces of information, rather than the entire source 
sequence. One such application is in bioinformatics (4), Q, 
in which a DNA sequence is stored as a binary string with 
relation to a reference sequence, with Is representing single 
nucleotide polymorphisms (SNPs) at those positions. In SNP 
calling, we are interested in learning whether there is a SNP at 
position i. Since we are not interested in any other information 
about the sequence, we would ideally like to accomplish this 
by accessing few bits in the compressed representation of the 
DNA sequence. In this specific instance, local decodability is 
strongly motivated, since decompressing the whole genome 
can be prohibitively expensive from a memory standpoint. 

Another example presents itself in the efficient storage of 
relationships among objects (e.g., relational databases 0). 
Given a collection of n objects, the relationships among these 
objects can be represented by an undirected graph on n ver¬ 
tices, with the presence or absence of an edge (i. j) signifying 
that objects i and j are related or unrelated, respectively (e.g.. 
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friendships in social networks). One can think of representing 
all graphs with n vertices by sequences of (”) bits representing 
all possible edges. A ‘1’ at a given position indicates the 
presence of that edge, and a ‘0’ indicates that it is absent. 
Thus, testing relationships between objects is accomplished 
by querying the value of the corresponding bit. As in SNP 
calling, it would be ideal to have a compressed representation 
of the graph which permits such queries upon accessing a 
small number of encoded bits. 

We remark that both applications referred to above involve 
a source that is inherently sparse - both SNPs and the number 
of relationships are small compared to the total length of 
the sequence. Motivated in part by this, our objective in 
this paper is to analyze the fundamental tradeoffs between 
access constraints and compressibility of sparse sequences, in 
the context of locally decodable compression. We consider a 
variable blocklength model, in which source sequences can be 
mapped to codewords of varying lengths, and that the decoder 
is informed of the codeword length at the start of the decoding 
process. 

Prior work on the problem of locally decodable source 
coding includes results on succinct data structures in the bit 
probe 0 , II and cell probe 0 , m complexity models. A 
widely studied problem in the bit-probe model is the static 
membership problem, which is closely related to the problem 
we consider. The bit-probe model for the static membership 
problem encodes subsets S C {1,2, ...,n} of size at most 
r into a data structure of fixed length such that queries 
of the form “Is i G S” for i G (1,2, ...,n} can be 
determined by probing (i.e., accessing) at most d bits in the 
data structure, either adaptively or non-adaptively. Buhrman et 
al. QH provided the lower bound i = Q^dr 1 1 ' d n 1 / d ), which 
remains the best lower bound for a general n, r, d. They also 
showed a scheme that achieves a blocklength t = 0(rd'n 1 t d ), 
with d' = d— 0(logr+loglogn) and d > 0(logr+loglogn). 
The interested reader is referred to fl2l for a comprehensive 
survey on several improvements to these bounds (for specific 
regimes of r and d) that have been proposed in the literature 
03 - 03 . Notably, fl3l considered d < 4 probes and showed 
that, for r = o(n), a blocklength t = o(n ) can be achieved 
by schemes using 3 adaptive probes or 4 non-adaptive probes, 
thereby settling a question posed in OH- By letting S denote 
the set of indices where a binary sequence has ones, the static 
membership problem considered by the bit probe model yields 
a fixed-blocklength, locally decodable representation of sparse 
sequences, and can therefore be viewed as a fixed-blocklength 


counterpart to the problem that we consider. 

Largely independent from the prior work on the bit probe 
model, locally decodable source coding has also received 
recent attention from the information theory community 021 - 
03 - Closely related to the present paper is the recent work by 
Makhdoumi et al. EQ, which considers the design of locally 
decodable source codes under the bit probe model for i.i.d. 
Bernoulli sources, with vanishing block-error probability. 

Our model differs fundamentally from those appearing in 
both 03] and m since we consider a variable-blocklength 
setting (to be defined precisely in Section Ell, which has not 
been previously studied. This is practically motivated because 
compressed file size is rarely fixed a-priori by the compression 
scheme, and file length is often recorded in metadata available 
to a decompressor. As we show in the sequel, our analysis 
of the variable blocklength case allows us to provide tight 
order-wise bounds on the average blocklength of the code 
in many cases — a problem that has remained open for the 
fixed-blocklength bit-probe problem for over 3 decades. Also, 
in contrast to 01 , we restrict ourselves exclusively to the 
lossless setting, in which the decoder must have zero (and not 
vanishing) probability of error, which is motivated by high- 
fidelity applications such as SNP calling. 

Our Contributions 

In this paper, we give upper and lower bounds on the 
average blocklength attainable by variable-blocklength com¬ 
pression schemes under local decodability constraints. These 
bounds are non-asymptotic in nature, and coincide (up to 
constant factors) in many cases. As a corollary, we give 
necessary and sufficient conditions on the number of bit-probes 
required to achieve competitively optimal compression perfor¬ 
mance, and briefly comment on connections to communication 
complexity. 

II. Notation and Problem Setting 

For an integer fc > 1, we employ the shorthand notation 
[k] = {1, 2,..., At}. We make frequent use of the conventional 
notations O(-), o(-), fi(-), w(-), ©(•). 

Throughout, we consider encodings of r-sparse binary vec¬ 
tors, which are simply sequences x n = (xi, X 2 , ■ ■ ■, x n ) £ 
{0,1}" having Hamming weight precisely r (we may assume 
without loss of generality that r < n/2). Our restriction to 
sequences of weight precisely r is primarily for convenience, 
since our arguments readily generalize to vectors having 
weight at most r. In some cases, we allow the sparsity 
parameter r to scale with n, in which case we write r n . 

The support of a source sequence x n is defined to be the set 
of nonzero coordinates, i.e. supp(ar") = {i £ [n] : xi = 1}. 
When referring to multiple distinct sequences, we will use the 
bracket subscript notation, i.e. x ™^, ar" 2 ), 

Letting (^) C (0,1}" denote the set of r-sparse binary 
vectors, we assume random vectors X n £ (^) are drawn 
uniformly from all ("j possibilities. A source code (i.e., 
compressor) c for r-sparse vectors is an invertible mapping 
c: (W) -»• {0,1}*, where {0,1}* = {0,1,00,01,10,...} 


denotes the set of all binary strings. For a codeword c(x n ) = 
(ci, C 2 ,..., ct) and a set of integers S C {1,2,... }, we let 
cs(x n ) = {ci : i £ S} denote the coordinates of the codeword 
indexed by entries in S. Letting ((h) denote the length of 
b £ {0,1}*, we remark that there are source codes for which 
the average codeword length is roughl\[] 

E [£{c(X n ))\ « log bits, (1) 

and this is best-possible, since the entropy of the source 
H{X n ) = log (”). Indeed, the naive scheme which lists the 
positions of each nonzero entry (requiring approximately log n 
bits each) is essentially optimal when r <C n. However, it 
is not clear whether such a source code admits a decoding 
algorithm that, for any specified index j £ [ n ], can recover 
bit Xj by probing a bounded number of bits in c(x n ). Thus, 
in the spirit of locally-decodable error-correcting codes l20l 
and the data structure counterparts in hd. m, we define a 
variable-length ( r , d , n)-locally decodable source code'. 

Definition 1. A (r, d , n)-locally decodable source code, or 
simply, an (r, d , n) code, consists of a mapping 

c : {0,1}* (2) 

with the property that, for each x n and j £ [n ], there exists a 
set S C {1,2,...} of size |S| < d for which Xj is a function 
of£(c(x n )) and cs{x n ). 

In other words, we can say c is a (r, d, n)-locally decodable 
source code only if there exists a corresponding ‘ (r, d, n)-local 
decompressor’ — i.e., an algorithm that takes as input a query 
index j £ [n] and the codeword length £(c(x n )), and returns 
the data bit Xj after accessing at most d bits of c(x n ). In 
light of this, we refer to the number d as an access constraint 
(or, decoding depth), since it bounds the number of encoded 
bits that the decoder probes before making a determination. 
In contrast to the fixed-blocklength settings that have been 
considered previously (cf. ifTTft . |H, 1201 ). Definition [j] does 
not preclude variable-length encoding schemes. As mentioned 
above, this is motivated by practice, where data structures are 
usually of variable length and any access protocol is cognizant 
of the encoded data’s length so that segmentation faults are 
avoided. Indeed, in computer file systems, a file is typically 
accessed after first reading metadata that describes the location 
and length of the file. 

Note that our definition of an (r, d, n )-local decompressor 
does not distinguish between adaptive or non-adaptive bit 
probes. That is, a decompressor can probe entries of c(x n ) in 
an adaptive manner (where codeword locations are accessed 
sequentially, and the positions accessed can depend on the bit 
values observed during previous probes), or in a non-adaptive 
manner (where codeword locations accessed are determined 
only by the query index j £ [n\ and the codeword length 
£(c(x n ))). When such a distinction is necessary, we will 
explicitly refer to adaptive and non-adaptive (r, d, n) codes. 

'Here and throughout, Iog(x) denotes the base-2 logarithm of x. 


III. Main Results 
A. Bounds on expected blocklength 

In this section, we present lower and upper bounds on 
the expected blocklength achievable by variable-length source 
codes obeying a local decodability constraint, and give suffi¬ 
cient conditions for them to coincide. Proofs can be found in 
Section IIVI 


Theorem 1. The expected codeword length of any ( r , d, n)- 
locally decodable code with adaptive bit-probes satisfies 


E[£(c(X"))]-M>(^) 


(C) 1/M+1) 



(3) 


Recalling the identity \m\ m ^ 00 m{x 1 / m — 1) = In a;, we 
note that for fixed n. r the lower bound 0 becomes 


lim 

d—f oo 


(cr +i ' 



( = 1 (") 

\ 4e / 4e \r J 


(4) 


Hence, we recover the information-theoretic lower bound 0 
(up to constant factors) in the absence of a local decodability 
constraint. On this note, an important consequence of Theorem 
[U is that it dictates how quickly d must scale with respect 
to n, r in order to accommodate encoding schemes that are 
near-optimal in the information-theoretic sense. In the next 
section we quantify this tension more precisely, and establish 
how large d must be in order to ensure competitive optimality. 
Before doing this, we discuss the tightness of ( 0 . 


Theorem 2. For any choice of r, d. n, there exists a non- 
adaptive (r, d, n)-locally decodable code c with average code¬ 
word length 


E [l{c{X n ))\ < 30 {rd + 1) ( (r + l) (r+1) 


l/(rd+l) 


(5) 


Two remarks are in order. First, we emphasize that Theorem 
Q]is a converse result for adaptive schemes, while Theorem[2]is 
an achievability result for non-adaptive schemes. We will see 
shortly that these bounds coincide (up to constant factors) in 
many cases, showing that adaptivity provides at most constant- 
factor improvement in these settings. Second, we note that both 
Theorem Q] and Theorem [2] are non-asymptotic in nature. That 
is, they hold for any choice of parameters r, d, n. However, 
results become most crisp when n —>■ oo, and r, d are functions 
of n. As a first example, we take n —>■ oo and r, d fixed 
(i.e., not depending on n). In this case, we find that the 
blocklength of an optimal sequence {c*} of (r, d, n) codes 
scales as E [f(c* (X n ))] = &{n r ^ rd+1 ' ) ). Hence, when r, d are 
fixed, performance scales poorly relative to the information- 
theoretic lower bound of @(log?r). 

As a second example, consider the setting where r n = n e 
and d n = 6 log n. Then it is a straightforward calculation 
using 0 and 0 to see that any optimal sequence {c*} of 
(r n ,d n ,n)- locally decodable codes will satisfy 


C ' 1 ( 2( 1_e )/ <5 


< E [l(c* (X ra ))] < c ^ 1/s 
~ 8n e log n ~ 


as n A oo, 


where C\ and C 2 are absolute constants. Thus, up to constant 
factors, the blocklength scaling behavior of optimal codes in 
this regime is Sn e log n, and the decoder will probe a fraction 
of the codeword proportional to 1 /r n in worst case. Contrast 
this with the trivial encoding scheme that simply stores the 
position of each ‘ 1 ’; the natural decoder based on binary search 
would require roughly log(r ra ) • log(n) probes in worst case. 

Similarly, if we parameterize n m = (™), r m = (1 + 
e)-m™(™) anc l d m = <51ogm, then as m —» 00 any optimal 
sequence {c^} of (r m , d m , ti m ) -locally decodable codes will 
satisfy 

Ci (2 1 /* - 1) < < C 2 2 2/5 . (6) 


This particular choice of parameters can be interpreted as 
encoding a random graph on m vertices with (1 + 
edges. Since is the threshold for connectivity, this graph 
is connected with high probability for e > 0. Now, querying 
whether two vertices are connected in this graph corresponds 
to querying a bit of X nm . In order to accomplish this in time 
that grows logarithmically in the number of vertices requires 
average blocklength of order rd = 0 (mlog 2 (m)). 

In the latter two examples, average blocklength scales 
r n d n = ©(log ( ")), which is within constant factors of the 
information-theoretic lower bound (i.e., competitively opti¬ 
mal). In both cases, we chose r n d n = fl(log( ? n )) in order 
to achieve this scaling. Thus, it is natural to ask: do there 
exist competitively optimal schemes with r n d n = o(log ("))? 
The answer to this question is negative, and is the focus of 
the next section. However, before we proceed, we unify the 
above examples under the following straightforward corollary 
of Theorems Q] and [2] 


Corollary 1. If log (”) = Cl(r n d n ) and d n = f2(logr n ), then 


any optimal sequence of (r„ 
{c* } satisfies 


, d ni n)-locally decodable codes 


E [%*(*"))] =0 r n d n 


l/(r„dn + l) N 


(7) 


B. Local Decodability and Competitive Optimality 

We now focus on the question raised at the end of the 
previous section, and give necessary conditions for compet¬ 
itive optimality (proofs can be found in the supplementary 
material). To this end, we define: 


Definition 2. For a sequence of integers { r r , } „ > i a sequence 
of encoders 


{ 0 , 1 }* 


is said to be competitively optimal if 

E[£(c„(X n ))] 


lim sup ■ .. 

n -¥00 log ( ) 


n > 1 


= 0 ( 1 ). 


( 8 ) 


(9) 


In other words, competitively optimal schemes attain com¬ 
pression rates within a constant factor of the information 
theoretic lower bound log ( r n ) for large enough n. 

From Theorem |T] it is possible to deduce the following 
necessary condition for competitive optimality: 







Theorem 3. If {c„} is a competitively optimal se¬ 
quence of (r n ,d n ,n)-locally decodable codes, then r n d n = 

In other words, we cannot expect to attain competitive 
optimality when r n and d n are simultaneously small relative 
to the source entropy (note the contrast to the sufficient 
conditions in Corollary [!}. This relationship can be somewhat 
complicated since the source entropy generally depends on 
both n and r n . However, when the source sequence is modestly 
sparse (i.e., r n = 0(n 1_e ) for some e > 0), then the explicit 
dependence on r n in Theorem [3] can be eliminated to obtain 
the following condition: 

Corollary 2. If r n = 0(n 1-e ) for some e > 0, then there 
exists a competitively optimal sequence of (r n , d n ,n)-locally 
decodable codes if and only if d n = fl(logn). 

In contrast to Corollary [2] if r n = 0(n), the information 
theoretic lower bound is log( r n ) = 0(n), and the identity 
encoding c n (x n ) = x n is competitively optimal, with all 
source bits being decodable with d n = 1 probes. 


IV. Proof Sketches for Main Results 


Due to space constraints, we only sketch the proofs of 
Theorems Q] and [2] Details are deferred to the supplementary 
material along with the proofs for Section UlI-BI 

Proof Sketch of Theorem [7f Our proof is inspired by 
that of HU Theorem 6], Let c be a (r, d, n)-locally decodable 
code. For a source sequence x n , let c q (x") denote the r/th 
coordinate of the codeword cand define the set 

T i — {( 9 , Cq: %(x£))) = k, and location q of c (x^) 
is accessed to determine Xj for some j £ supp(a;^)}, 


where we have abused notation slightly by letting Xj denote 
the jth coordinate of sequence xfy. Note that each Tf is a 
subset of [fc] x {0,1} of size at most rd, since |supp(a:"-j)| = r 
and the decoder makes at most d probes in response to a query. 
Also note that for i 7 ^ i ', T.f % Tfi. To see this, assume the 
contrary, that if' C Tf) for some i f i'. Let the encoded 
source word be x'f ,,. If we now query the value of Xj for 
j £ suppfrr”^) \ suppfrr^j), we see that the decoder will 
make an error, establishing the contradiction. 

Since for fixed k, the Tf s are not subsets of one another, 
an application of the LYM inequality [22) yields 

#{i: £(c(x?^)) = k\ < max ( ) for each k. (10) 

^ ’ v<rd Y V J 

In light of (flOt . the average codeword length must satisfy 


E[f(c(AT n ))] > 


M(n,r,d ) 

E 


k =1 


m&XyCrd 


a 



ail 


where M (n, r, d) is the largest integer satisfying 


M(n,r,d )-\-1 


E 


max 

v<rd 


> 


M(n,r,d) 

> E 

k=l 


max 

v<rd 


Now define the probability distribution 


Q(k) = 


max- 


O 


a 


for 1 < k < M(n,r,d) (13) 


and Q(M(n,r,d) + 1) = 1 Qik). Since Q(k) < 

Q(k + 1) for k < M(n,r,d) by definition, we can conclude 


M(n,r,d) +1 

k ■ Q(k) > 

fe=1 


M(n, r,d) + 1 
2 


(14) 


Toward evaluating d, we need the following technical 
estimate, which is proved in the supplementary material. 


Lemma 1. For all M, v > 1, 

' 2k \ + 2 + ^) v+1 


M 

E max 

i<v 

k=1 ~ 


< 2 V 


(■v + 1 )! 


(15) 


Identifying M ■£- M(n,r,d) + 1 and v ■£- rd in (fl5] >. the 
first inequality in (ITTI i can be rearranged to conclude 

l/(rd+l) 


M (n, r,d) + 3 > 



- 1 


' rd + 1 
2e 


(16) 


Recalling (IT4l) proves the desired inequality. ■ 

Proof Sketch of Theorem [2} The proof is by a random 
coding argument, but it is important to note that standard 
typicality arguments are not applicable here since they do 
not support local decodability. Briefly, the idea behind our 
encoding scheme is to first encode some information about 
supp(c(a;")) into the codeword length, and then carefully 
encode the remaining information so that bit Xj can be 
recovered by computing the binary AND of d encoded bits. A 
precise description of the codebook generation and decoding 
procedure is given below. An example and the analysis are 
postponed to Appendix IA-BI 

Codebook Construction: For k = rd+1, rd+ 2,... choose 

( k \ 

a subset Sk Q [n] of size uniformly at random from 


all such subsetfj^. For each j £ Sk , choose a subset Tyk Q [k] 
of size d independently and uniformly from all such subsets. 
All subsets are made available to both encoder and decoder. 

For a sequence x n £ (^), let k(x n ) denote the smallest 
integer k such that the following two conditions hold: 

(Cl) supp^ 71 ) C Sk', and 

(C2) T jtk % U iesup p( :E n ) T iife for all j £ S k \ supp)^ 71 ). 

Encoding procedure: A sequence x n £ (f r f ) is encoded to 
a codeword c(x n ) of length k(x n ) satisfying 


s UPP(c(x )) 0j^supp(a) n )-^t,/c(a; n ) ■ (12) 


In other words, x n is encoded to a vector of length k{x n ), 
which has l’s in all positions j £ Ti k(x n ) if an d on ly if Xi = 1- 
Decoding procedure: On observing the length of codeword 

c(x n ) = (ci, C 2 , ■ • ■, Q( c ( x »))), determine bit x :j as follows: 

(1) If j ^ 5 /(c ( x »)), declare Xj = 0; else 

(2) If j £ S t ( c ( lB)) , declare Xj = A ieTjMa n ,0 , where ‘A’ 
denotes binary AND. 


( 12 ) 


2 Floor and ceiling operators are omitted for clarity of presentation. 











By the nature of the codebook construction, it is clear that 
the decoder (i) will never make an error; and (ii) satisfies 
the non-adaptive d-local decodability constraint. The analysis 
of the average codeword length is omitted due to space 
constraints, and can be found in Appendix IA-BI ■ 

V. Concluding remarks 

We provided bounds for the blocklength scaling behaviour 
of (r, d. n) locally-decodable codes that are order-wise tight 
for many regimes of r, d, and n, although determining the 
tight constant in these bounds is still an open problem. We 
also showed that in contrast to the fixed blocklength setting 
(cf. El), adaptivity of probes provides no essential advantage 
in our regime of variable length source coding. In conclusion, 
we mention two variations on our main results: 


A. Compression with block errors 

In El, the authors allow for vanishing block-error prob¬ 
ability in decoding. Although we only considered error-free 
encodings, the proof of Theorem Q] readily extends to incor¬ 
porate block-error probability as follows: Letting x n denote 
the decoder’s estimate of the sequence x n given codeword 
c(a; n ), the block error rate is defined to be Vr{X n X n }. 
Now, Theorem Q] continues to hold for any (r. d. n) code with 
block-error rate e by simply replacing the quantity ("J with 
(1 — e)("). Indeed, this follows by considering only those 
sequences that are correctly decoded and making the same 
substitution in (fl2l > in the proof of Theorem |T] 


B. Connection to communication complexity 

It is known that the bit-probe model has applications 
to asymmetric communication complexity ED. To draw an 
analogous connection to our setting, consider an asymmetric 
communication complexity model El in which Alice (the 
user) has i £ [n]. Bob (the server) has S C [n] of size r, and 
they wish to compute the membership function 




1 if ieS 
0 otherwise. 


(18) 


We now enforce that the function / must be computed 
under a SpeedLimit paradigm, which proceeds as follows. 
Communication starts with Bob sending a speed limit message 
to Alice consisting of some z bits, which limits the length of 
any of her messages to z bits. Bob’s subsequent messages 
consist of 1 bit. After the initial round, Alice and Bob 
communicate over d roundel to evaluate /. The setting arises 
in practice where a server imposes upload bandwidth limits on 
users it serves (e.g., to maintain quality or fairness of service). 

Note that our scheme in Theorem [2] provides a communica¬ 
tion protocol to compute / under the SpeedLimit paradigm. 
Bob is essentially given a source sequence x n , which he stores 
as c(x n ). Alice is given the index i of the source bit that must 
be decoded, and must do so by making queries to Bob. Bob 
begins by sending £(c(x n )) to Alice, using logf(c(x")) bits. 


3 To be consistent with the rest of the paper, a communication round consists 
of one message by Alice and a response by Bob. 


Alice then sends messages rrij, j £ [d] of logf(c(:r n )) bits 
each. In response to message rrij, Bob sends back c mj (x n ). 
Alice then announces / to be the AND of the d bits she has 
received from Bob. Therefore, from Theorem [2] 


Corollary 3. There exists a deterministic communication 
protocol for computing the function f as in El under the 
SpeedLimit paradigm for which the speed limit z and 
number of communication rounds d satisfy 


E[2 J ] < 30(rd + 1) 


(r + l)( r+1 ) 



l/M+i) 
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Appendix A 

Supplementary Material 


In this section, we provide further details for the proof 
sketches given in Section [HI] For convenience, we recall the 
following standard inequalities which will be used repeatedly 
throughout the proofs without explicit mention: 



A. Details for proof of theorem [7] 

The LYM inequality used in proving the lower bound (O 
is given below for convenience. 

Lemma 2 (LYM inequality ll22l l. Let U be a u-element set, 
let A be a family of subsets of U such that no set in A is a 
subset of another set in A and let a m denote the number of 
sets of size m in A Then 


Em- 1 - (21) 

m —0 \rnj 

The second key inequality was Lemma Q] which is restated 
below for convenience. 

Lemma [l] For all M, v > 1, 
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Proof of Lemma [Tf We begin the proof by splitting the 
bound into two cases: 

Proposition 1. For v > 1 


Next, if M > v + 1, then 
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(30) 


establishing the second claim. 

In light of Proposition Q] it is sufficient to check that 

(3(« + l)!) 1/(,,+1) 1/iV+1) < 2 (m + 2 + ^±1 


for 1 < M < v in order to prove (1221) . Since 

{3{v + l)\) 1/{v+1) (^^ A <3 1/(l,+1) ^^(2e) 


(31) 


M/{v+ 1) 
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and the RHS in (l32l > is a convex function in M, we can show 
(fTil holds by verifying 
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for M = 0 and M = v + 1. It is straightforward to check that 
this is the case. ■ 
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for 1 < M < v 
for M > v + 1. 
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Proof: Note that if M < v, then 



B. Details for proof of Theorem \2} 

The random encoding scheme described in Section IIII-AI is 
illustrated in Figure Q] The only thing remaining is to analyze 
the performance of this scheme: 

Performance Analysis: To show a bound on the expected 
codeword length, fix an arbitrary sequence x n and define the 
events 


We will prove by induction on M that XEi Ck) — |( 2 jv/)- 
The base case M = 1 is trivial, so by the inductive hypothesis, 
we have 



However, we can write 

/2(M + 1)\ _ (2 M + 2)(2 M + 1) (2M\ ^ o (2M\ 
i M + l ) ~ (M + l) 2 V M ) ~ 3 V M ) ’ 

completing the proof of the first claim. 


SkA = {supp(a; n ) C S k } 

&k ,2 — ^ klzEsupp(cc™) Ti^k for all j tz Sk \ supp(x )}. 

By independence of the sets used in the codebook construc¬ 
tion, we have 

£c WC+ 1 ))] (34) 
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= E kPr{£ k ,i D S k , 2 } (1 -Prjfj-,1 n£j i2 }), 

fc>rd+1 j=rd-\-1 

where Eg [•] denotes expectation over the ensemble of random 
codebooks. Importantly, we note that (l34l > is a decreasing 
function of Pr{£fc.i n £k/ 2 } for each k. Therefore, in order 













Therefore, we have 
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The challenge of the proof is to now carefully bound ( l39l ) and 
(E§. Toward this goal, we further bound Yi{£k,i T £k, 2 } as 
follows: 


Pf{^fc,i H Sk.'i} > 
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Hence, 



Fig. 1. Encoding of 2-sparse source sequences of length n = 12 using a 
codeword of length k = 10, with at most d = 3 bit probes. As a result, \Sk | — 
-^q-j- = 4. The source sequences ,..., a;^ represent those 

that satisfy condition (Cl) of the encoding criterion for Sk = {2,3, 5,6}. 
Condition (C2) is only satisfied by x ^, x ^, x™ Q >, , which are encoded as 
shown. If the bit to be decoded j E [n] \Sk = {1,4, 7,8, 9,10,11,12}, then 
the decoder outputs 0 without probing the bits of the codeword. If j E Sk, 
then the decoder probes positions Tj of the codeword and returns the AND 
of the bits (shaded blocks correspond to l’s, unshaded blocks signify 0’s). 


to upper bound (l34l ). we will lower bound Pi{£k,i IT£fc ;2 }- To 
that end, observe that 


Pr{£ M n 4, 2 } > C r , d k rd , (44) 

where we have defined C r ,d = ((")(r + 1 ) r+1 (erd) rd ) 1 for 
convenience. 

Now, using the inequality (1 — x) < e~ x , we can upper 
bound (f34b with ( 144b as 
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Pr{£ M n S k , 2 } = Pr{ffc,i} Pr{£ fc;2 |£ fci i} (35) 

= i ?y Pr {4, 2 |4,i}, (36) 

where the conditional probability Pr{£fc !2 |£fc,i} can 
bounded from below by a simple union bound: 
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Since the function ue~ u is monotone increasing on (0,1) and 
monotone decreasing on (1, oo) with a maximum of 1/e, we 
can bound the sum in ( l49b as 
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The integral in (l50l > can be bounded as follows: 
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Proof: Abbreviating C := C Tt< i, consider the change of 
variables z = 
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where (l55l > follows because T(a:) < 1 for x < 2. ■ 

To finish the proof, we use Lemma [3] and the integral upper 
bound on (|49]> to conclude 
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where (l57l > is a (loose) upper bound assuming rd > 1, which 
follows from elementary algebra and the definition of C rJ i. 

Since all sequences in (^) are equally probable, linearity 
of expectation ensures the existence of a code c which satisfies 
© as desired. 


Remark 1. We have made no significant attempt to optimize 
the multiplicative constant in ©. In general, the given proof 
reveals that this factor of 30 can be replaced by a function 
of rd that is bounded by 30 when rd = 1 and is upper 
bounded by 2 + e as rd grows large. We conjecture that the 
achievability scheme proposed in the proof of Theorem \2\ can 
yield a multiplicative factor as small as T which is 

strictly less than 1, at the expense of more careful intermediate 
bounds. 


C. Proof of Theorem [T] 

Suppose {c„} is a competitively optimal sequence of 
(r n ,d n ,n)- locally decodable codes, and define e n according 
to 


r n d n + 1 = e n log 



(58) 


Then Theorem |T] and the definition of competitive optimality 
imply that there is some constant K for which 
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for all n sufficiently large. Since e(2 1 / e — 1) + 1 oo as e \ 0, 
this implies that there is a constant K' > 0 such that e n > K' 
for all n sufficiently large, proving the claim. 


D. Proof of Co rollary \2\ 

To prove the “only if’ direction, note that Theorem[3] asserts 
that there must be a constant K > 0 such that 
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Thus, by the assumption that r n = 0(n 1_e ) for some e > 0, 
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d n > A'log — = fl(logn). (63) 

r n 

To prove the “if’ direction, suppose there are positive 
constants K, e such that for n sufficiently large 


d n > Klogn (64) 

r n < n l ~ e . (65) 


Since reducing d n can only adversely affect performance, we 
can assume without loss of generality that 
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for n sufficiently large. As a consequence, we have 
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An application of Theorem [2] completes the proof. 
























